1. Field of the Invention
This invention relates to a method, system, and processor-executed software for authenticating and validating still images and videos (imagery) captured by a smartphone or other digital camera device. The method not only enables detection of image tampering, but also enables verification of the time the image was taken, its location, and other information that may be used to determine the authenticity and validity of the imagery.
The method involves using metadata associated with the image capture to authenticate and verify the images and videos, and protection of the metadata by public/private key encryption. The metadata may include not only time and date, but also other data such as camera settings (aperture, shutter speed, focal length, and so forth), camera orientation and movement data, and context information such as sounds or words captured contemporaneously with the imagery, the direction in which the image is taken, and signals from nearby cell towers or WiFi hotspots.
The imagery itself is watermarked with a unique identifier that is embedded in the image using a symmetric key generated by the smartphone or other camera device, and the watermarked image, metadata, and symmetric key are digitally signed and uploaded or transmitted to a central authentication and verification service for storage in a database upon authentication of the digital signatures of the watermarked imagery, metadata, and symmetric key. The central authentication and verification service, which may be a cloud based service, enables third parties to authenticate and verify submitted imagery that corresponds to imagery stored in the authentication database.
2. Description of Related Art
Currently cell phone cameras are ideal for capturing images and video on a moment's notice, at the drop of a hat, any place, any time. Their revolutionary ubiquity also puts cell phone cameras everywhere all the time. However, although cell phone imagery can ignite wide interest, the validity of cell phone imagery is subject to uncertainty owing to the potential for losing the ground truth of the image, its time of capture or its place of capture as a result of deliberate image, time or GPS doctoring or subsequent image, time or GPS mishandling or misadventure. A realistic video of Sasquatch or the Loch Ness Monster can be created inside a 16-year-old's bedroom in Queens, N.Y., and yet find its way to a popular news website. A video of a candidate's speech can be altered by an opponent's campaign staff. Alien creatures can find their way into images purporting to be transmitted by the Mars rovers. For this reason, the usage of cell phone images or video to ascertain ground truth is limited. While cell phone images and video have great value for enjoyment, they have much more limited value as legal evidence, detective information, or scientific data.
Nevertheless, the very ubiquity of cell phone imagery opens an enormous opportunity to gather valid information on all matters of human interest, on an unprecedented scale. If such imagery could be captured in such a way as to enable anyone viewing the imagery to authenticate and validate the imagery, the value of the imagery would be substantially increased.
It has long been known to encrypt at least portions of electronic documents in order to authenticate the source of the images, and enable detection that the document has been tampered with. However, such techniques are not adequate to authenticate digital imagery captured by portable devices such as smartphones. In the case of imagery, it is not sufficient merely to authenticate the source of the imagery, or that the image has been tampered with after creation. One needs to know how the imagery was created, and the circumstances of its creation. For example, one might wish to know whether a picture of the Loch Ness monster was actually captured from a moving boat in outdoor light.
Furthermore, whereas the authenticity of a document is primarily of interest to the recipient of the document or a limited group of persons affected by the document and who can be given keys to authenticate the document, the authenticity and/or validity of an image captured by a smartphone camera may be of interest to a much larger audience with access only to the imagery itself, and not associated data necessary for authentication and validation.
These problems have only been addressed in parts. Both digital image watermarking and image authentication techniques have previously been proposed, but these techniques do not address the underlying truth of the imagery, i.e., whether the imagery actually shows what it purports to show, and are generally unsuitable for implementation on a mass scale, i.e., by the vast numbers of ordinary smartphone or digital camera device users, and even larger numbers of person who might have subsequent access through the Internet to the imagery captured by the smartphones or digital camera devices.
An example of a prior image authentication method is found in U.S. Pat. No. 6,005,936, which discloses a digital camera that generates authentication information data from a first region of the image, encrypts the authentication information data from the first region and embeds it in the second region. Tampering can be detected by recovering and decrypting the embedded authentication information data from the second region and comparing the decrypted authentication data with the original authentication data in the first region. This method allows detection of image alterations. However, it does not authenticate the original image, i.e., it does not ensure that the original image is an image of what it purports to show, and that it was taken at its apparent location and at the time it appears to be taken.
Another example of a prior image authentication method is the watermarking technique is disclosed in U.S. Pat. No. 8,107,668, which also involves embedding information in the imagery, but which embeds the information of watermark throughout the imagery in a way that that is difficult or impossible to detect (known as staganographically embedding), thereby increasing the robustness of the watermarking by increasing the difficulty of tampering while at the same time enabling dissemination of the watermarked image. Again, however, this patent does not address underlying issues involving verification of image content or context, or of providing an integrated system that enables implementation on a mass scale, i.e., by the general smartphone-owning and Internet-image viewing public.
Finally, a company called EvidencePix Systems, Inc. has proposed a system that provide secure transmission of images, but limited to the context of security systems that send intruder alerts to subscribers. The transmissions are in the form of encrypted digital images to which time and location are annotated, as described in U.S. Pat. Nos. 7,146,479 and 7,535,352, but the system does not provide validation of the image capture process, and in particular the ability to verify that the time and date associated with an image are authentic and properly associated with the image, or a way for a general audience to access and authenticate/validate the images.